Chapter 8 Multimodal Integration : fMRI , MRI , EEG , MEG

This chapter provides a comprehensive survey of the motivations, assumptions and pitfalls associated with combining signals such as fMRI with EEG or MEG. Our initial focus in the chapter concerns mathematical approaches for solving the localization problem in EEG and MEG. Next we document the most recent and promising ways in which these signals can be combined with fMRI. Specifically, we look at correlative analysis, decomposition techniques, equivalent dipole fitting, distributed sources modeling, beamforming, and Bayesian methods. Due to difficulties in assessing ground truth of a combined signal in any realistic experiment—a difficulty further confounded by lack of accurate biophysical models of BOLD signal—we are cautious to be optimistic about multimodal integration. Nonetheless, as we highlight and explore the technical and methodological difficulties of fusing heterogeneous signals, it seems likely that correct fusion of multimodal data will allow previously inaccessible spatiotemporal structures to be visualized and formalized and thus eventually become a useful tool in brain imaging research.